Efficient Dynamic Resource Allocation Using Nephele in a Cloud Environment

Today, Infrastructure-as-a-Service (IaaS) cloud providers have incorporated parallel data processing framework in their clouds for performing Many-task computing (MTC) applications. Parallel data processing framework reduces time and cost in processing the substantial amount of users’ data. Nephele is a dynamic resource allocating parallel data processing framework, which is designed for dynamic and heterogeneous cluster environments. The existing framework does not support to monitor resource overload or under utilization, during job execution, efficiently. In this paper, we have proposed a framework based on Nephele, which aims to manage the resources automatically, while executing the job. Based on this framework, we have performed extended evaluations of Map Reduce-inspired data processing task, on an IaaS cloud system and compared the results with Nephele framework.